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LETTER OF TRANSMITTAL 


House OF REPRESENTATIVES, 
CoMMITTEE ON SCIENCE, SPACE, AND TECHNOLOGY, 
Washington, DC, June 26, 1992. 
Members of the Committee on Science, Space, and Technology 

I am pleased to transmit the report of the Task Force on the 
Health of Research. The Task Force was convened at my request in 
May 1991, in order to investigate widespread reports of “‘stress” in 
the federal research system, and to define a more active role for 
the Committee in the forging of science ‘policy for the national 
good. The Task Force was composed of staff representatives of full 
Committee and all subcommittees. This report summarizes their 
deliberations and conclusions. It does not represent either findings 
or recommendations adopted by this Committee. 

For the last fifty years, the United States has supported the 
world’s most innovative and productive scientific research system, 
a system designed to create new knowledge and technology that 
can help, in the words of Vannevar Bush, to “insure our health, 
prosperity, and security as a nation in the modern world.” Today, 
however, the United States faces wide-ranging societal crises and 
challenges, in our educational system, our environment, our manu- 
facturing sector, our health care system, our inner cities, our finan- 
cial institutions, even our system of government. . 

This paradox—growing knowledge, accompanied by growing soci- 
etal crisis—implies a complex, nonlinear relationship between ad- 
vances in knowledge and advances in society. More specifically, it 
suggests either that we are not adequately using the knowledge 
that we already have, or that we are not sufficiently producing the 
knowledge that we actually need. Of course, both of these condi- 
tions may be partly true. 

This report offers some strategies and insights that may help us 
begin to solve the knowledge paradox. It challenges us to create 
better linkages between the research that we authorize, and the 
national goals that we seek. The report is intended to be a begin- 
ning, a tentative agenda for Committee action and future debate. I 
commend it to your attention. 

Sincerely, 
GEORGE E. Brown, dr., 
Chairman. 
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REPORT OF THE TASK FORCE ON THE HEALTH OF 
RESEARCH 


SUMMARY 


This report outlines broad strategies by which the Committee on 
Science, Space, and Technology (SST) can strengthen its oversight 
of the federally funded research portfolio, while crafting policy to 
improve the linkages between federally funded research programs 
and national goals such as energy security, environmental protec- 
tion, innovation in high technology manufacturing, expanding the 
knowledge base, and educating future scientists and engineers. The 
report is not intended to be comprehensive; rather, it seeks to 
expand the current science policy dialogue, which has been too nar- 
rowly focused on funding levels, and not sufficiently concerned 
with the structure of the research system. 

The Task Force on the Health of Research views national goals 
as the motivating force behind federal support for research. This 
perspective suggests two lines of analysis that must underlie feder- 
al research policy: 

(1) For a given national goal, what research is most neces- 
sary? 

(2) How can we best conduct such research? That is, what 
mechanisms for administering, performing, and evaluating re- 
search create the optimal pathways from research to goal at- 
tainment? 

To answer these questions, the Committee will need to examine 
and evaluate traditional science policy assumptions and recom- 
mend new approaches; expand the range of experts upon whom it 
calls for advice to include the users of research, rather than just 
the performers; and establish programs that compare and assess al- 
ternative methods of research administration and performance. In 
this context, the Task Force recommends two nested lines of action 
aimed at creating more explicit linkages between national goals 
and federally supported research programs: 

(1) Strengthen mechanisms for setting government-wide re- 
search-policy goals, and for oversight of the federal research 
portfolio, by exploiting Committee jurisdiction over the Office 
of Science and Technology Policy; 

(2) Integrate performance assessment mechanisms into the 
research process using legislative mandates and other meas- 
ures, to help measure the effectiveness of federally funded re- 
search programs. 

Implementing the recommendations of this report will require a 
long term, strategic approach by the Committee that will probably 
include hearings, pilot projects, and legislative initiatives embrac- 
ing the jurisdictions of most or all of the SST subcommittees. The 
report itself is not a detailed recipe; rather, it is designed to serve 
as the starting point of a constructive dialogue and evolutionary 
process that can strengthen the Committee’s ability to forge sci- 
ence policy for the national! good. 
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PROBLEM STATEMENT 


The present system of research in the United States was estab- 
lished in the two decades following World War II. This period was 
characterized by policy consensus, minimal international competi- 
tion, and a relatively small and homogeneous research community. 

Today, there is a widespread perception that the federally funded 
research system is under stress. From the perspective of research- 
ers dependent on federal support, this may simply mean that there 
is more competition for available funding, or that mission-oriented 
research budgets are insufficient to fulfill agency mandates. From 
the perspective of policy makers, stress may be manifested by dis- 
cord between the promised benefits of research and a society beset 
by a range of seemingly intractable economic, environmental, and 
social problems. 

Research policy designed forty years ago may no longer be suita- 
ble for addressing the problems of today’s world. Traditional disci- 
plinary and agency boundaries, unsophisticated models of innova- 
tion and economic benefit, and ideological approaches to federal in- 
volvement in the research process must be reconsidered. 

The Administration’s budget request establishes an implicit re- 
search agenda, and Congress responds. The Committee on Science, 
Space, and Technology has reinforced the status quo by addressinz 
policy issues and reauthorization on an agency-by-agency and year- 
by-year basis, and by providing what often amounts to a rubber 
stamp for agency budgets, agendas, and policies. 

The Committee must consider new strategies for addressing in- 
creasingly urgent policy dilemmas and societal challenges. Such 
strategies should be rooted in the development of explicit linkages 
between the conduct of federally funded research and the achieve- 
ment of national goals. The need for change is clear. 
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INTRODUCTION 


The U.S. system of scientific, engineering, and mathematical re- 
search is widely perceived to be under stress. The scientific commu- 
nity, science-policy makers, and the press have each sought to ex- 
plain the sources and potential impacts of this stress, often arriv- 
ing at highly divergent interpretations. From a Congressional per- 
spective, concern over the health of the federally funded research 
system demands a reconsideration of the proper role of the federal 
government in supporting research.} 

Alleged causes of this stress include: ? 

(1) Inadequate federal resources to support a growing aca- 
demic research community at levels which the community be- 
lieves are necessary (for research and training); 

(2) Inadequate federal resources to support mission-oriented 
research (i.e., research designed to achieve established policy 
objectives) at federal agencies; 

(3) Increased demand for new interdisciplinary research pro- 
grams to investigate complex and/or high profile research 
problems (e.g. global climate change, AIDS); 

(4) Lack of mechanisms and criteria for setting priorities in 
federally funded research both within and between disciplines, 
fields, or programs; 

(5) Increasing proportion of federal research funds spent on 
“big science” projects; and 

(6) Declining private-sector investment in research, resulting 
in fewer research jobs and increased demand for federal sup- 
port of research that had formerly been carried out by private 
industry. 

Perceived symptoms of stress in the research system include: 3 


1 The Task Force defines the federally funded research system to include all nondefense re- 
search supported by federal funds, regardless of performer. The system encompasses all basic 
and applied scientific and engineering research, as well as development of facilities and equip- 
ment for research, but does not include development of products and processes when they 
become directly linked: to commercial applications. A comprehensive reconsideration of the fed- 
eral role in this system is contained in U.S. Congress, Office of Technology Assessment, Federal- 
ly Funded Research: Decisions for a Decade, OTA-SET-490 (Washington, DC: U.S. Government 
Printing Office, May 1991). 

2 For university research, see, for example: Science: The End of the Frontier? a report by Leon 
M. Lederman, President-Elect, to the Board of Directors of the American Association for the 
Advancement of Science, January 1991; Robert M. White, “Too Many Researchers, Too Few Dol- 
lars,” Issues in Science & Technology, vol. 7, Spring 1991, Pp. 35-37; Erich Bloch, “Optimists, 
Skeptics, and Realists: Other Views of the Research ‘Crisis’,” in Science and Technology Poli 
Yearbook 1991, M.O. Meredith et al., eds. (Washington, DC: AAAS, 1991); and Roland W. 
Schmitt, “The Sources of Discontent in Academic Research,” presented at the annual meeting, 
American Association for the Advancement of Science, Feb. 9, 1992, Chicago, IL. For mission- 
oriented research, see, for example: W.D. Kay, “The Politics of Fusion Research,” Issues in Sci- 
ence and Technology, vol. 8, Winter 1991/92, pp. 40-46; E.S. Rubin, L.B. Lave, and M.G. Morgan, 
BoA we: Climate Research Relevant,” Issues in Science and Technology, vol. 8, Winter 1991/92, 
pp. 47-55; Expert. panel on the role of Science at EPA, Safeguarding the Future: Credible Sci- 
ence, Credible Decisions, U.S: Environmental Protection Agency EPA/600/9-91/050, March 1992. 

3 In addition to references in footnote 2, see, for example, U.S. Con; , Office of Technol 
Assessment, Competing Economies: America, Europe and the Pacific Rim, OTA-ITE-498 (Wash- 
ington, DC: U.S. er ament Printing Office, Ocho 1991); National Academy of Sciences, The 
Government Role in Civilian Technology (Washington, DC: National Academy Press, 1992); 
Glenn McLaughlin and Richard E. Rowberg, Sieg 4 Between Federal Research and Develo 
ment Funding and Economic Growth,” CRS Report for Congress 92-211 SPR, February 21, 1992; 
John F. Ahearne, “Why Federal Research and Development Fails,’ Resources for the Future 
Discussion Paper EM88-02, July, 1988; Bruce Stokes, “High Tech Tussle,” National Journal, 
rh a thee 1990, pp. 1838-1342; and Leon Jaroff, “Crisis in the Labs,” Time, August 26, 1991, pp. 
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(1) Declining competitiveness of U.S. high-technology prod- 
ucts in international markets; 

(2) Inability of mission-oriented research programs at many 
sat agencies to achieve policy-relevant results or statutory 
goals; 

(3) Increased competition for funding among researchers, and 
increased time spent on administrative, versus research, activi- 
ties; 

(4) Neglect of undergraduate-level science education at many 
universities because of an excessive emphasis on pursuit of re- 
search funding and performance of research; 

(5) Increasingly vocal debates over the appropriate role of 
federal funding in applied civilian research; 

(6) Publicity about issues such as misconduct in research, 
academic earmarking, misuse of indirect cost reimbursements, 
conflicts of interest, and animal testing. 

This report accepts the premise that these and related concerns 
are indicative of “stress’’ within the research system. Regardless of 
the merit of these and other commonly cited problems, it is clear 
that neither policy makers nor scientists are satisfied with the im- 
plementation of science policy today. In this context, this report 
seeks to outline broad strategies by which the Committee on Sci- 
ence, Space, and Technology can strengthen its oversight of the 
federally funded research portfolio, while crafting policy directed 
at improving the linkages between federally funded research pro- 
grams and national goals. 

To broaden and deepen the science policy dialogue, this report 
questions basic assumptions about the operation and goals of the 
federal research system. This dialogue, on the whole, has been too 
narrowly focused on funding levels, and not sufficiently concerned 
with how to increase the ability of federally supported research to 
contribute to the welfare of the nation. The emphasis on funding 
levels indicates a complacency with institutional structure and per- 
formance that is unwarranted and perhaps dysfunctional for the 
times. The ongoing federal budget crisis should be viewed as a mo- 
tivation for the Committee to examine fundamental science policy 
assumptions—still rooted in a cold war mentality—and consider 
new approaches to implementing a science policy for the 1990s. 

A principal objective of this report, then, is to propose specific 
strategies that the Committee may wish to pursue to enhance the 
contribution of federally funded research to national goals such as 
increased economic competitiveness, enhanced human health, 
energy security, environmental protection, and expansion of the 
knowledge base. Two principal strategies are identified: 

(1) Strengthen mechanisms for setting government-wide re- 
search-policy goals, and for oversight of the federal research 
portfolio, by exploiting Committee jurisdiction over the Office 
of Science and Technology Policy (OSTP); 

(2) Integrate various performance assessment mechanisms 
into the research process using legislative mandates, in order 
to help measure the effectiveness of federally funded research 
‘programs. 


== 
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BACKGROUND 


Current federal science policy was articulated during and shortly 
after World War II.* Federal funding for civilian research was seen 
as a means of supporting research that the private sector could not, 
or would not, support. Universities became the principal basic re- 
search performers, Federal labs carried out mission-oriented tasks, 
and the private sector pursued most of the non-defense applied 
Research and Development (R&D). Disciplines and sectors tended to 
operate in isolation from one another. Cooperation and collaboration 
in most civilian research was left to informal agreements, not to formal 
joint ventures strategically planned to achieve targeted goals. A 
fundamental assumption of this system was that it should operate in a 
more-or-less laissez-faire manner, with minimal governmental inter- 
ference. A principal tenet of U.S. science policy has been that 
researchers are most qualified to judge and administer the work of the 
research community, and, in the case of basic research, that attempts 
to control direction or outputs of research would stifle creativity, 
innovation, and, ultimately, technological development. 

The role of scientific research during World War II, as exempli- 
fied by the Manhattan project, created a new national legitimacy 
for scientists (especially physicists) as contributors to national well- 
being. With the exception of national-prestige “research” projects 
such as Apollo and the War on Cancer, the U.S. civilian research - 
agenda, both basic and applied, has been driven as much by the 
imagination and expertise of scientists as by any particular set of 
policy goals. Culture and policy have bestowed highest status to 
basic research and its performers, located mainly at universities. 
Thus, scientific leaders and research models are largely defined at 
our research universities, and are strongly influenced by discipli- 
nary and departmental structure. This is de facto science policy 
today, and may grow stronger as industry reduces its basic re- 
search capability.® 

Federal mechanisms for funding and administering research 
have remained relatively unchanged for 45 years.® However, there 
appears to be a growing mismatch between the demands and expec- 
tations of the research community (forged during the 1950’s and 
1960’s) and the goals of policy makers (which reflect current politi- 
cal, economic, and societal pressures). Fiindamental problems 
facing our society require interdisciplinary research approaches 
rather than reliance solely on traditional disciplinary paradigms. 


* Everybody’s benchmark is Vannevar Bush, Science: The Endless Frontier, (Washington, DC: 
US. Office of Scientific Research and Development, July 1945, and reissued as a National Sci- 
ence Foundation report, 1960). Also see reports associated with this Committee: National Acade- 
my of Sciences, Committee on Science and Public Policy, Basic Research and National Goals, a a 
report to the Committee on Science and Astronautics, 'S. House of Representatives Nib 
ton DC: March 1965); and U.S. Congress, House Committee on Science and Fe ione ia 
Force on Science Policy, A History of Science Policy in the United States, 1940-1985 (Washing- 
ton, DC: U.S. Government Printi ea: a eine 1986). 


5 Private industry is increasi Migs to invest in research that offers no promise of 
short-term payback; see: John well, Fes Matter Physics in a Market nomy,’ 
Physics "Today, May, 1992, pp. 40-47; “Wasted ’ The Economist, September 3, 1988; pp. 


64-66; John Marko, “A ‘A Corporate Lag i in apache Funds is Causing Worry,” New York Times, 
January 22, 1990, p 

®For Biectinticn: and analysis of the “Federal research system”—agency sponsors, research 
performers, the bu rocess, and the participation of various constituencies—see OTA, Feder- 
ally Funded Researc fs shone 3-5, op. cit., footnote 1. 
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There is a recognized need for closer cooperation between private 
and public sector scientists, as well as between U.S. and foreign sci- 
entists, and natural and social scientists. In attacking such funda- 
mental problems, boundaries between basic and applied research 
will be ever more difficult to demarcate. Social science research, 
once a marginal part of the federal research agenda, is now 
thought by some to be a necessary component of a balanced re- 
search portfolio (as well as a prerequisite for understanding how 
that portfolio can be constructed and modified to achieve societal 
goals).? Finally, and perhaps most conspicuously, it is becoming in- 
creasingly clear that maintaining the world’s preeminent (and 
most expensive) federal research system is not, in and of itself, ade- 
quate to insure economic vitality. Overall, there is a growing rec- 
ognition that the organization and conduct of research may not be 
appropriately designed to address the explicit goals of society (as 
articulated by policy makers), or to supply policy makers with the 
information they require to make informed science- or technology- 
based decisions.® 


RETHINKING SCIENCE POLICY 


Perhaps we must reconsider the ways in which Federal agencies 
define their roles in the research system. It is unlikely that stress in 
the research system can be relieved simply by increasing research 
funds.!° For example, major increases in federal biomedical re- 
search support in the 1980s seem to have had, if anything, the op- 
posite effect.11 This is probably because a substantial fraction of 
any new funding goes to universities, which stimulates graduate 
enrollments, and leads to growth in the number of researchers that 
inevitably outstrips the ability of the federal government to pro- 
vide adequate support. However, the debate over funding level ig- 


7 For Sagres see John Ziman, “A Neural Net Model of Innovation,” Science and Public 
Policy, Vol. 18, February 1991, pp. 65-75; and Howard Newby, “One Society, One Wissenschaft: 
A 21st Century Vision,” Science and Public Policy, Vol. 19, February 1992, pp. 7-14. Both papers 
challenge the “linear model” of science translating into technology, which provides the impetus 
for social and economic progress. The latter in addition sees social science as central for under- 
standing the interactive process linking scientific excellence and technological innovation to eco- 
nomic and social well-being. 

8 The correlation between research and economic vitality is a cornerstone of post-War federal 
science we According to one influential report: “If we are to remain a bulwark of democracy 
in the world, we must continually strengthen and expand our domestic economy and foreign 
trade. A principal means to this end is through the constant advancement of scientific knowl- 
edge and the consequent steady improvement of our technology” [(John R. Steelman, Science 
and Public Policy, A Report to the President (Washington, D.C.: U.S. Government Printing 
Office, 1947) v. 1, pp. 3-4]. A more recent observer suggests that there is no evidence of a posi- 
tive correlation between the strength of our academic basic research capability, and our econom- 
ic vitality; see Derek Bok, “Reconciling Conflicts—The Challenge for the University,” in DS. 
Zinberg (ed.), The Changing University, (Dordrecht: Kluwer Academic Publishers, 1991), pp. 15- 


® This recognition is reflected in the recent activities of mainstream rast Weed and sci- 
ence-advocacy organizations. For example, the Carnegie Commission on Science, pahoc ig ane 
Government will soon issue a report on “Linking Science and Technology to Societal a 
The Council of Scientific Society Presidents has also proposed a study on “Establishing Prior- 
ities for Scientific Research” which would address the issue of priority-setting in the context of 
societal goals. 

10 One of the few representatives of the academic research community to acknowledge this 
point is the President of Rensselaer Polytechnic Institute, Roland Schmitt. For example, see his 
talk “Science—The Frontier of Challenge: Adjusting to Finite Resources,”” NAS/NAE/NSB Sym- 
posium, February 15, 1991, Irvine, CA. ‘ 

11 OTA, Federally Funded Research, chap. 2 and p. 223 (box 7-D) op. cit., footnote 1. Barbara J. 
aia ee Funding: The Eternal ‘Crisis’”, Science, vol. 250, December 21, 1990, pp. 
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nores the broader context. The real question is whether available 
resources are being allocated in a manner that can best achieve na- 
tional goals. Yet this question has never been addressed in a com- 
prehensive manner. 

Traditionally, the federal research mandate has called for the sup- 
port of research that is neglected by the private sector. For 1991, 
this included federal funding for about 50 percent of all basic re- 
search performed in the U.S., and about 30 percent of the applied 
research.!2 This has translated into a leading role for U.S. univer- 
sities as performers of research. In 1991, 38 percent of all federal, 
non-defense research funds went to universities. Intramural federal 
laboratories were the second largest research performer, receiving 
28 percent of all non-defense funds. Industrial performers received 
11 percent, and university-administered National Laboratories re- 
ceived 10 percent. How do we know that this is an optimal distribu- 
tion? We don’t. 

In attempting to rethink the fundamental nature of federal sci- 
ence policy, it must be recognized that the community of federally 
funded researchers shares many attributes with other interest 
groups that receive federal support: it resists change; it seeks addi- 
tional resources as a cure for. internal stress; it develops political 
(i.e., subjective and partisan) strategies to promote its agenda and 
demonstrate the need for special treatment; it unselfconsciously 
gives its own values primacy; and, in particular, it strives to show 
that it is an essential contributor to the national interest. Further- 
more, the research community has certain political advantages 
over other interest groups. One is a product that is reputedly 
unique—objective data upon which the nation can make policy de- 
cisions and build economic growth. This creates an apparent moral 
superiority over other interest groups that are shackled to more ob- 
viously partisan or pecuniary agendas. 

Another political advantage enjoyed by the scientific community 
is the privilege of being trusted to judge and police itself. This 
privilege applies not merely to expert peer reviewers, but to wit- 
nesses at Congressional hearings, administrators in federal agen- 
cies, and members of agency advisory committees. Thus, in the 
postwar science policy paradigm, advice is sought from the produc- 
ers of knowledge. The users of science are rarely canvassed for 
their views on research policy (and when they are, they appear to 
be less disturbed by “‘stress’’ within the research system than by a 
ae of connectedness between research and practical applica- 
tion). 

It is clear that scientists are especially qualified to judge the sci- 
entific merit of research. Although many research programs offer 


12 National Science Board, Science and Engineering Indicators—1991, Tenth Edition, (Wash- 
ington, D.C., U.S. Government Printing Office, 1991). 

Indeed, many economists and representatives of industry suggest that our research system 
is just about the only sector of our economy that is in eter od good health. For example, see 
recent testimony by Robert M. Solow at hearing on dogs fs Options for Promoting Economic 
Growth,’ Committee on Science, Space, and Technology, May 5, 1992; Keith Pavitt, “What 
Makes Basic Research Economically Useful?” Research Policy, Vol. 20, ‘1991, pp. 109-119; Na- 
tional Research Council, Research on the Management of Techno : Unleas the Hidden 
fmaPehi fine Advan (Washin nm, DC: National Academy Press, 92). Lewis M. Branscomb, 
Sa Does America N: a Technology Policy?” Harvard Business Review, March-April 1992, pp. 
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the explicit promise of achieving certain nonscientific payoffs, the 
research programs which Congress authorizes and funds rarely in- 
clude mechanisms to judge progress toward such goals. Thus, the 
success or failure of a research program is typically judged in 
terms of “scientific excellence.” Few programs are modified or ter- 
minated for failing to achieve promised policy goals. 

Despite the acknowledged stresses within the research system, 
there has been little or no change in the fundamental assumptions 
and principles that have been guiding federal science policy.1* This 
policy is captive to its own history, which asserts: 

Basic research is the source of fundamental knowledge 
that eventually leads to innovation, technology development, 
and economic growth. 

(2) Basic research should be carried out primarily at univer- 
sities, in isolation from political constraints or applications-ori- 
ented management. 

(3) Individual-investigator research is the most productive of 
new ideas. 

(4) “Scientific excellence,” as judged by peer review, is the 
best criterion for determining which projects should receive 
funding. 

(5) Federally funded applications-oriented research should be 
managed by individuals with technical expertise, i.e. scientists 
and engineers. 

(6) Applied research restricts creativity and serendipitous 
discovery. 

(7) Academicians must perform cutting-edge research in 
order to be effective educators. 

(8) The connection between research results and societal ben- 
efits is indirect, but we accept as an article of faith that more 
high-quality research will necessarily translate into more bene- 
fits for society at large. 

(9) Mission-oriented research can provide objective data to 
guide public policy decision-making in a wide range of areas, 
including the general economy, energy, and environment. 

(10) Traditional scientific disciplines form the most natural 
and appropriate framework for funding, administering, per- 
forming, and evaluating research. 

A major failing of U.S. science policy has been the absence of in- 
stitutional mechanisms designed to test the validity of these and 
related assertions. There are three fundamental reasons for this 
failure. First, the U.S. research system was designed primarily by 
and for scientists, and these assertions serve to preserve the auton- 
omy and legitimacy of the research community. Second, it is the 
researchers themselves who are called upon to evaluate a system 
which they have little incentive to alter. Third, the economic per- 


14 The Lederman report, cited in footnote 2, is the most prominent recent iteration; another is 
a speech by Frank Press, “Science and Technology Policy for a New Era,” byes at the Na- 
tional Academy of Science’s 129th annual meeting, Washington, DC, April 27, 1992. Conversely, 
it is worth noting that there have been isolated calls for change and predictions of coming stress 
for at least the 8 ee 25 years Pino Weinberg, Reflections on Big Science (Worcester, MA, 
The Heffernan , 1967); and ph Sanders and Fred R. Brown, ., Science and Technolo- 
gy: Vital National Assets (Washington, DC: Industrial College of the Armed Forces, 1966), espe- 
cially pp. 93-94]. However, this message has only recently begun to find an audience, as stress 

grown within the research system, the federal budget, and society as a whole. 
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formance of the United States in the 1950’s, 60’s and 70’s has been 
construed as a vindication of these science policy principles. (It 
must be emphasized, however, that the economic preeminence of 
the U.S. after World War II was virtually a foregone conclusion be- 
cause the production capacity of most of our economic competitors 
had been destroyed.) 

Since we have lived with only one post-war science policy para- 
digm, we cannot know whether a different one might have been 
more or less effective. What we do know is that a paradigm predi- 
cated on a distinctive blend of rugged individualism and unfettered 
competition led the United States to world leadership in Nobel 
prizes, numbers of publications, and numbers of patents. We also 
know, however, that several of our economic competitors have 
adopted radically different science policy principles (Japan being 
the shining example), and have had great success in linking re- 
search with national goals, especially in the area of sustained eco- 
nomic growth.!5 There are lessons to be learned as well as impart- 
ed. Perhaps, too, it is time to experiment with our own approach. 
Such experiments should be carefully defined and modular, to 
avoid throwing out the good with the bad, but they should test a 
wide range of policy alternatives. 


SETTING PRIORITIES 


Priority setting has become a catchphrase in science policy dis- 
cussions. The debate over priority setting tacitly reinforces the sen- 
timent that the major science policy problem of the day is inad- 
equate funding. The lack of adequate resources to support all 
worthy research projects and programs demands that priorities be 
set and choices be made—this is now accepted by much of the re- 
search community. Within specific disciplines it is generally ac- 
knowledged that the researchers themselves are best suited to set 
priorities. !6 

Choices between disciplines, however, often cannot be made on 
the basis of scientific merit, because there is no clear framework 
for comparison.!7? Thus, it is commonly accepted that nontechnical 
criteria, and political considerations, will play a role in determin- 
ing priorities between disciplines. All the same, the research com- 
munity has made some attempts to set priorities across disciplines, 


15 Perhaps the most obvious indicator of differing national science policies is the variation in 
distribution of research funds by socioeconomic objective. For example, the U.S. devotes a much 
greater proportion of its R&D budget to health, and a much smaller proportion to energy and 
industrial development, than do most other industrialized nations. See International Science and 
Technology Data Update: 1991 Special Report NSF 91-309 (Washington, D.C., National Science 
Foundation, 1991). For brief descriptions of science and technology policies of several other na- 
tions, see previously cited Office of Technology Assessment reports Federally Funded Research 
and Competing Economies, footnotes 1 and 3. 

16 The most recent example comes from the National Academy of Sciences, Space Studies 
Board, Setting Priorities for Space Research: Opportunities and Imperatives (Washington, DC: 
National Academy Press, 1992). 

17 This was clearly revealed in hearings held three years ago by this Committee. See U.S. 
Congress, House Committee on Science, Space, and Technology, Subcommittee on Science, Re- 
search, and Technology, The Hearings on Adequacy, Direction and Priorities for the American 
Science and Technology I bag 101st Cong., Feb. 28-Mar. 1, 1989, (Washington, DC: U.S. Govern- 
ment Printing Office, 1989). These hearings nee Members’ frustrations with evaluating 
witnesses’ arguments for increased resources. petaining. disciplines and research programs was 
the chief reason given, with only vague reference to “enhancing economic competitiveness.” The 
question of national goals was not prominent in the discussion, but became a prime motivation 
for the aforementioned OTA study on Federally Funded Research 
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especially in the case of “big science” projects, which are seen as 
potentially threatening to funding for individual-investigator re- 
search. !8 

If priority setting is undertaken predominantly as a response to 
tight funding, then it threatens to entrench further the existing 
science policy model by reinforcing disciplinary boundaries and fo- 
cusing on funding as the single most important determinant of a 
successful national research effort.!9 Thus, the much-lauded efforts 
of various disciplinary communities to set priorities may be seen as 
an indication of fiscal responsibility, while having the effect of cir- 
cumventing a more fundamental reconsideration of the tenets of 
science policy. 

As an alternative to priority-setting exercises driven by fiscal 
concerns, and circumscribed by disciplinary boundaries, science- 
policy makers could view priorities as derivatives of societal or 
policy goals. This approach suggests two lines of analysis and 
action in setting priorities: 

(1) For a given national goal (e.g., energy independence; ex- 
pansion of the knowledge base), what research is most neces- 
sary? 2° 

(2) What mechanisms for administering, performing, and 
evaluating research create the optimal pathways from research 
to goal attainment? 

These two areas of concern occupy the core of the recommenda- 
tions for policy action made in this report. These recommendations 
can be viewed as priority setting from a science-policy perspective, 
rather than from the more common resource-allocation perspective. 


PoLicy RECOMMENDATIONS: LINKING RESEARCH To NATIONAL 
GOALS 21 


U.S. science policy has traditionally focused on facilitating the 
performance of research, while neglecting to evaluate the overall 
organization of the research system, and the ways that research re- 
sults are integrated into society and translated into societal bene- 
fit.22 In this context, the Committee may wish to consider a funda- 
mental reformulation of science policy principles, with the view 
toward exploiting research as a tool designed to achieve national 


18 For example, during the 1991 debate over appropriations for NASA’s Space Station Free- 
dom, 12 scientific societies cosigned a letter to members of the Senate Appropriations Commit- 
tee stating opposition to continued funding for the station. In 1989, the Industrial Research In- 
stitute conducted a survey of member corporations in which the corporations ranked five major 
federally funded ‘‘big science” projects in order of potential benefit to the U.S. industrial base. 

19 The latest example is Albert H. Teich, “Discussions of Setting Science Priorities Are Filled 
With Misunderstandings,” The Chronicle of Higher Education, Jan. 22, 1992, p. A52. 

20 See testimony from a hearing on “Setting Priorities in Science,’ held by the Subcommittee 
on Science, Committee on Science Space and Technology, on April 28, 1992. Witnesses unani- 
mously endorsed the concept that priority-setting could only be carried out in the context of 
clearly stated goals. 

21 There will be considerable debate over whether such a linkage should be attempted. The 
arguments against it are various, familiar, and murky, ranging from the difficulty of agreeing 
on goals, to the fear that explicit linkages will stifle creativity. The arguments for such a link- 
age are clearer, but may conflict with conventional science policy tenets: we should try to 
ensure value returned for federal tax dollars expended, by striving to maximize the capability of 
federally funded research programs to meet the expectations and needs of policy makers and 
society. 

22 Perhaps the most comprehensive (and controversial) revisionist view of science policy pre- 
sented to date is Deborah Shapley and Rustum Roy, Lost at the Frontier: U.S. Science and Tech- 
nology Policy Adrift, (Philadelphia, PA, ISI Press, 1985). 
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goals, rather than as a black box into which federal funds are de- 
posited, and from which social benefit is somehow derived. 

The flip side of stress within the research system is a growing 
perception that federally funded research is not sufficiently con- 
tributing to long-term national goals such as economic competitive- 
ness, human health and environmental protection.?3 In spite of the 
promised contribution of research, both’ basic and applied, to high 
technology innovation, long-term economic vitality, and standard of 
living, the U.S. seems less able to translate scientific and techno- 
logical advance into societal gain than several of our international 
competitors. Similarly, the promise of mission-oriented research to 
address a wide range of policy issues in areas such as energy, envi- 
ronment, and public health, has frequently not been realized.?* Al- 
though the failure to achieve such long-term national goals is not 
necessarily the fault of the research system per se, neither has 
there been a successful attempt to link research programs explicit- 
ly to goals in a manner that would optimize the policy-relevance of 
the research while maintaining the highest possible degree of sci- 
entific excellence. 

There are many reasons why such linkages do not now exist. In 
part, the complex organization of the Executive Branch makes it 
difficult to view the federal research system as a whole, and the 
many Congressional committees with jurisdiction over some por- 
tion of the research portfolio make it difficult to impose any sort of 
uniform policy standards.?* Furthermore, the historical dominance 
of science policy by researchers themselves has often precluded any 
requirement to explain and justify the research system to Congress 
and other users of research. Indeed, the Committee has traditional- 
ly been engaged in creating policy for science and scientists, which 
may not be the same as creating mechanisms to improve the appli- 
cation of research to societal problems. 


23 These goals are explicitly and implicitly set by Congress and by the Executive Branch, and 
they presumably reflect, to some degree, the desires of the populace, as filtered through elec- 
tions, the press, etc. However, mechanisms to measure progress toward goals (or to assess the 
reasonableness of goals) are rarely implemented or even considered. For some specific cases, see 
Technology and Economic Performance: Organizing the Executive Branch for a Stronger National 
Technology Base, report of the Carnegie Commission on Science, Technology, and Government, 
September, 1991; Mark Andersen and Kosta Tsipis, “Federal R&D policy and its implications for 
U.S. economic competitiveness: A compendium of facts,’’ Massachusetts Institute of Technology, 
Program in Science, Technology and International Security, unpublished ms., November, 1989; 
Kelly Day and Vernon Ruttan, “The deficit in natural resources research,” BioScience, vol. 41, 
January, 1991, pp. 37-40; Leslie Roberts, ‘Counting on science at EPA,” Science, vol. 249, 
August 10, 1990, pp. 616-618; and George E. Brown, Jr. and Radford Byerly, Jr., “Research in 
EPA: A Congressional point of view,’ Science, vol. 211, March 27, 198i, pp. 1885-1390. 

24 The previously cited OTA report on Competing Economies outlines the relationships be- 
tween U.S. science and technology policy, and twenty years of decreasing competitiveness. Also, 
see: Erich Bloch, “‘Basic Research and Economic Health: The Coming Challenge,’’ Science, vol. 
232, May 2, 1986, p. 595-599; and Susan L. Sauer (ed.), Science and Technology and the Changing 
World Order, (Washington, DC: AAAS, 1990). A conspicuous example of a mission-oriented re- 
search program that is failing to achieve desired goals is the high-level nuclear waste disposal 

rogram; see: Board on Radioactive Waste Management, Rethinking High-level Radioactive 

laste Disposal, (Washington, DC. National Academy Press, 1990). For examples of other pro- 

ams that have failed to achieve anticipated goals, see: Leslie Roberts, ‘Learning From an Acid 
Rain Program,” Science, vol. 251, March 15, 1991, pp. 1302-1305; and Eliot Marshall, ‘Artificial 
Heart: The Beat Goes On,” Science, vol. 253, August 2, 1991, pp. 500-502. 

25 In the Executive branch, 10 departments and independent agencies have civilian research 
budgets in excess of $100 million, while nine House and six Senate committees have jurisdiction 
over major portions of the research budget. Also, see National Academy of Public Administra- 
tion, ond Distrust: Building Bridges Between Congress and the Executive, (Washington, DC: 
NAPA Panel on Congress and the Executive, January 1992). 
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To create a more rigorous and socially-responsive science policy, 
a necessary first step is to define goals toward which the research 
should be expected to contribute. Such goals may be general and 
long-term such as: achieving national energy security, enhancing 
environmental protection, improving human health, increasing the 
productivity and profitability of high technology industries, main- 
taining a healthy research infrastructure, expanding the knowl- 
edge base, educating future scientists, and creating a scientifically 
and technologically literate work force. Goals may also be more fo- 
cused and/or shorter-term such as determining the impacts of 
changing climate patterns on regional agriculture or sea level, or 
recommending a site for nuclear waste disposal.”® 

Such goals may seem obvious (many of them have been generally 
accepted for years), but in fact they serve more often as rhetorical 
devices, exploited by policy makers and researchers alike, than as 
guides to the implementation of research strategies. Furthermore, 
the goals articulated by Congress may not be shared by the scien- 
tists and engineers who conduct the research aimed at achieving 
these goals.27 In addition, the criteria by which progress toward 
goals is measured may differ for policy makers and researchers. 
Whereas scientific excellence is the foremost criterion by which re- 
search is judged within the research community, such excellence 
does not guarantee policy relevance or potential application to 
technological innovation. It may be necessary but not sufficient. 
The creation of new scientific knowledge does not guarantee the 
utility, diffusion and implementation of that knowledge.?® In fact, 
the barriers that separate researchers from policy makers (de- 
signed to protect researchers from political interference) may also 
act to impede diffusion and implementation. 

Congress is a part of this problem. The federal research system is 
organized in such a way that congressional committees and execu- 
tive agencies are most directly responsive to the voices of the re- 
searchers that they support. Committees and agencies become ad- 
vocates for the programs (and scientists) under their jurisdiction, 
rather than for the achievement of policy objectives and national 
goals that do not fall neatly within jurisdictional bounds. 

There is a broad range of concrete actions that the Committee 
could consider to link national goals more closely to research per- 
formance. Some of these actions focus on creating and strengthen- 
ing mechanisms of knowledge diffusion (ie., the “links” them- 
selves—pathways from research performance to knowledge utiliza- 
tion). This area of activity has been explicitly emphasized in the 
fiscal 1993 Views and Estimates of the Committee on Science, Space, 


26 In addition to clearly articulating its own goals, the Committee may want to ask the pro- 
posers of research programs to state the goals that they are addressing (and to suggest metrics 
of success), in order to determine if the Committee’s aims are consistent with the aims of those 
who will carry out the research. 

27 For example, a recent analysis of the Global Change Research Program suggests that the 
questions that policy makers most want to see answered are often very different from those 
which the researchers are pursuing. See Joint Climate Project to Address Decision Makers’ Un- 
certainties, Report of Findings (Washington, D.C., Science and Policy Associates, Inc. 1992). Also, 
see the statement by Dr. William O. Baker in: Committee on Science and boca Me Seminar 
on Research, Productivity and the National Economy, (Washington, D.C., U.S. Government 
Printing Office, 1980), pp. 60-72. 

28 Eg, Branscomb, “Does America Need A Technology Policy?” op. cit., footnote 14. 
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and Technology,?° a report to the House Budget Committee that 
recommends increased federal coordination of technology policy; 
expanded industry-government consortia and other partnerships; 
and expanded use of manufacturing technology extension centers. 

Another way to help define research-policy linkages without 
interfering excessively in the research process is to identify and ex- 
ploit new sources of information that can contribute to the science 
policy process. The Committee could expand the range of witnesses 
on which it calls for expert advice, to include a greater emphasis 
on the users, rather than the performers of research.®° Users of re- 
search may include members of the business and legal communi- 
ties; educators; state and local policy planners; public interest 
groups; journalists; the military; and other researchers. Congress 
and the Executive Branch of course are also users of research, and 
the Committee must be willing to assess the utility of research pro- 
vided to the federal government for policy-making purposes. Re- 
search performers may also be able to identify the potential users 
of their own work. 

In addition to the users of research, there is a growing number of 
science policy analysts who conduct “research on research,” and 
have much to contribute to the science-policy dialogue. It should be 
noted that these analysts are often considered to be “outside” the 
research system, yet it is they who are conducting rigorous (‘‘scien- 
tific’) study of the practice and use of science in society. As well as 
seeking a wider range of witnesses, the Committee could encourage 
greater diversity and independence in the membership of agency 
advisory committees. The dominant role of scientists and engineers 
on these committees may represent an inherent conflict of interest, 
while excluding critics of the status quo. 

The Committee could also acquire new information about the 
process of knowledge diffusion in today’s research system. For ex- 
ample, the Committee could ask CRS or OTA to do a long-term 
study on mechanisms of knowledge transfer in fostering the appli- 
cation and utilization of “breakthrough” technologies. Such a study 
might focus on current examples, such as Fullerenes, high-temper- 
ature superconductors, and global change. What are the feedback 
paths between basic research, applied research, and knowledge uti- 
lization? How can Congress act to facilitate the development and 
utilization of such paths? 

Most importantly, the Committee can act to strengthen its abili- 
ty to establish government-wide science-policy goals, and to institu- 
tionalize mechanisms for measuring progress toward such goals. 
These two areas of action are discussed in more detail below. 


29 Views and Estimates of the Committee on Science, Space, and Technology on the Fiscal Year 
1993 Budget for epee R&D, 102nd Cong., Second Session (Washington, DC; U.S. Government 
Printing Office, 1992). 

30 The ability and/or willingness of Congress to use this advice is a separate problem, and 
beyond the scope of this report. Certainly the availability of sound advice does not guarantee its 
wise use; see Lawrence E. Lynn (ed.), Knowledge and Policy: The Uncertain Connection (Wash- 
ington, D.C.: National Academy of Sciences, 1978). 
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1. Exercising jurisdiction over the Office of Science and Technology 
Policy *} 


The Office of Science and Pecan ey. Policy (OSTP) and the Fed- 
eral Coordinating Council on Science, Engineering, and Technology 
(FCCSET) have the combined authority and responsibility to devel- 
op and implement coherent, government-wide science po icy.32 The 
Committee’s jurisdiction over OSTP and FCCSET gives it unique 
leverage over the science-policy-making capability of the Executive 
Branch. No other House Committee has a mechanism for exercis- 
ing influence over the entire civilian research portfolio. The Com- 
mittee should therefore strive to increase its level of oversight and 
strengthen its guidance of OSTP and FCCSET. In parallel, the 
‘Committee should seek to strengthen OSTP’s mandate to formu- 
late, coordinate and implement research policy. 

In most of the 15 years since its creation, OSTP has not been a 
strong player in the Executive Branch. The influence of OSTP may 
be growing, however, because of the increasing demand for a more 
explicit U.S. technology policy, and a need for a coordinating mech- 
anism for large, interdisciplinary, interagency research programs. 
The Committee can strengthen OSTP’s standing and importance 
within the Executive Branch by treating it as a command center 
for implementing and evaluating all major research policy deci- 
sions. This may require additional funds, additional statutory au- 
thority, and, in particular, additional oversight by the Committee. 
The goal is to help move science-policy making from the current, 
ad hoc, agency-by-agency, OMB-dominated process that exists 
today, to a more strategic process oriented toward the conduct, 
goals, and users (not just the performers) of research. 

The Committee’s jurisdiction over FCCSET should be prudently 
exercised. FCCSET is the mechanism through which the Adminis- 
tration coordinates research programs that are not limited by the 
boundaries of agencies and disciplines. FCCSET membership in- 
cludes the heads of all federal research agencies, and as such it has 
the potential (thus far exercised on a limited basis) to engage in 
government-wide science-policy planning. The Administration is 
using the FCCSET process to initiate inter-agency research pro- 
grams in five areas of particular national interest.°* In this sense, 
the Executive Branch is ahead of the Congress, and the Committee, 
in terms of exploring new ways to implement research policy in the 
context of evolving national goals.°4 The FCCSET “cross-cuts” may 
represent a major component of a shifting science-policy paradigm. 


_ 51 Of course, exercising jurisdiction is a means, not an end. The end is better science policy 
judged ANE contribution to achieving the types of goals discussed in the previous section. 

32 OSTP and FCCSET were established by the Science Policy Act of 1976 (PL 94-282). The act 
states that, among other responsibilities, OSTP shall ‘“‘seek to define coherent approaches for 
applying science and technology to critical and emerging national and international problems 
and for promoting coordination of the scientific and technological responsibilities and programs 
of the Federal departments and agencies in the resolution of such problems.” FCCSET shall, 
among other responsibilities, “provide more effective planning and administration of Federal 
scientific, engineering, and technological programs.” _ 

33 High performance computing; advanced materials and processing; biotechnology; global 
change; and mathematics and science education. Note that only the High Performance Comput- 
ing Program (P.L. 102-94) and the Global Change Research Program (P.L. 101-606) are author- 
ized in specific statutes. . 

34 However, the National Climate ks wap Act of 1978 (P.L. 95-367) provided for such a cross- 
cutting program of climate research, to coordinated through OMB in a manner analogous to 
the way that FCCSET coordinates such programs today. 
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The process by which the FCCSET cross-cuts are selected is still 
somewhat mysterious. Administration sources emphasize a 
“bottom-up” process through which cross-cutting initiatives are 
proposed by researchers and research administrators, and consid- 
ered and adopted by FCCSET membership. There is no clear set of 
policy goals toward which the cross-cuts are directed. Criteria for 
success, and procedures by which FCCSET initiatives may become 
institutionalized as National Research Programs, have not been 
fully aired or analyzed. Similarly, criteria for evaluation of Nation- 
al Research Programs have apparently not been established. The 
Committee must exercise close oversight, implement mechanisms 
to measure the effectiveness of the cross-cuts, and work to increase 
Congressional input into the FCCSET process. At the same time, 
the Committee must be sensitive to the importance of flexibility, 
creativity, and informality in the FCCSET process. These attributes 
(as opposed to political inertia and agency “turf” protection) can fa- 
cilitate the effective linking of research and goals, by permitting an 
evolutionary approach to research policy planning. The Commit- 
tee’s involvement in this process could begin with oversight of the 
recently initiated cross-cut in advanced materials and processing. 

The objective of strengthening OSTP’s science-policy capability is 
not to consolidate power in one office; rather, it is to create a mech- 
anism for making science policy that responds to the broad needs 
of the nation, not only to the narrower needs of individual agen- 
cies, disciplines, institutions, or scientists. Under OSTP, FCCSET 
offers a forum for bringing together directors of all research agen- 
cies. The Committee could introduce legislation requiring that this 
group convene on an annual or semi-annual basis in order to devel- 
op a coherent science policy rooted in explicit, articulated goals. Al- 
ternatively, the Committee could create a new administrative 
structure (separate from FCCSET) that would require this type of 
inter-agency planning activity. In either case, the interests of each 
agency would be represented, but the process of developing nation- 
al science policy would be coordinated, and potentially strategic. 
Furthermore, the federal research budget should be developed as 
an outgrowth of this coordinated approach, rather than emerging 
piecemeal from the arbitrary bounds of agencies and their research 
constituencies. ?5 

The process for achieving this type of policy making could begin 
with OSTP drafting a document which sets out explicit, strategic 
research goals for federally funded research. These goals should be 
developed independently of consideration of budgetary constraints 
and agency boundaries. Goals should be reviewed by the Commit- 
tee, and evaluated by experts within and outside the research com- 
munity, including the intended users of the research results. Anal- 
ysis and assessment of goals should be revisited on a periodic (e.g., 
biennial) basis. Development of a research portfolio designed to 
meet these goals within existing budgetary limits could then be un- 


35 The need for this type of coordinated Executive-Branch ed tg to science policy was 
strongly endorsed by National Institutes of Health Director Bernadine Healy at an April 7, 1992 
hearing on “Setting Priorities in Science,” held by the Subcommittee on Science, Committee on 
Science, Space, and Technology. Healy’s comments were conspicuous enough to be reported in 
Science & Government Report, vol, 22, May 1, 1992, p. 1. 
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dertaken through the FCCSET (or alternative) process, and imple- 
mented via existing agency procedures. 


2. Integrating performance assessment into the research process 


Federal policy -has effectively created or facilitated the creation 
of new research opportunities, while providing resources for a 
growing research community and supporting the education of 
future generations of scientists. Federal policy has been less effec- 
tive, however, in monitoring the progress of research programs 
toward particular goals once the programs are underway. The need 
for performance assessment mechanisms is suggested by a number 
of observations: 36 

¢ Cost overruns and tue ongoing political debate over “big sci- 
ence” projects; 

e Discord between promised and actual performance of federal- 
ly funded research programs; 

° Difficulty of turning high quality science into policy-relevant 
and/or economically beneficial science in federally funded re- 
search programs; 

e Failure to identify, modify, and shut down unsuccessful re- 
search programs; 

e Lack of criteria for measuring the effectiveness of ongoing 
research programs; 

¢ Difficulty in meeting mandated goals in federally funded re- 
search programs. 

Federal research policy decisions must not be based solely on 
input criteria (i.e., funding based on the promise of research), but 
on concrete outcomes as well. The Committee could require that 
the programs it authorizes be subject to performance assessment 
that is fully integrated into the administration of the research 
process. For such mechanisms to be effective, the assessments 
would have to be carried out by persons or organizations independ- 
ent of the research performers, who are knowledgeable about both 
the scientific and policy aspects of the research. Performance 
guidelines could be built into authorizing legislation,?? or deter- 
mined through an independent evaluation process. Furthermore, 
there would have to be a clear statutory mandate to redirect or ter- 
minate programs that are not making sufficient progress toward 
stated goals. Performance assessment could have the following 
positive effects on research authorized by the Committee: 

e Agencies and researchers alike would be more likely to set 
poe and recommend budgets that are realistic and achieva- 
e, 


36 E.g., U.S. Congress, Congressional Budget Office, Large Non-defense R&D Projects in the 
Budget: 1980-1996 (Washington, DC: CBO Papers, July 1991); U.S. Congress, Congressional 
Budget Office, “Federal Investment in Intangible Assets: Research and Development,” How Fed- 
eral S; os for Infrastructure and Other Public Investments Affects the Economy, (Washing- 
ton, DC: study, July, 1991), pp. 73-101; dialogue between Senator Albert Gore and Prof. 
Lewis Branscomb in: Senate Subcommittee on Science, Technology, and Space, Committee on 
Commerce, Science, and Transportation, Heart on National Science and Technology Policy, 
101st Co , September 28 and 29, 1989, (Washington, DC: U.S. Government Printing Office, 
1989), pp. 86-97; also see footnotes 23, 24 and 27. 

31 For example, the Committee linked the availability of authorized funds for the Supercon- 
ducting Super Collider (SSC) to the achievement of a series of technical and administrative crite- 
ria in the SSC authorization bill passed by the House of Representatives in 1990 (H.R. 4380). 
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¢ The Committee would have a source of information to evalu- 
ate programs under its jurisdiction that is independent from 
the actual performers or supporters of the research; 

¢ Weaknesses and strengths of the research portfolio would be 
more quickly revealed; 

¢ Research performers would have a greater incentive to work 
toward promised goals of their programs. 

The Committee could initiate this effort by authorizing one or 
two pilot programs, such as those recommended for NASA mis- 
sions, or for the multi-agency Global Change Research Program.?8 
It must be emphasized that there are significant obstacles to inte- 
grating performance assessment into the research system. Political- 
ly, there will be resistance from the federally funded research com- 
munity to any suggestions of interference in the current research 
process. There will also be a justifiable fear of even further in- 
creases in paperwork and red tape. Indeed, any assessment process, 
while serving the Committee’s oversight responsibilities, should be 
designed to lessen the bureaucratic responsibilities of the research 
community, preferably by replacing other audit activities. 

More daunting than political resistance to performance assess- 
ment are the technical obstacles. Because policy-oriented assess- 
ment has not been a part of the research process in the past, its 
implementation must be both gradual and flexible.°® In general, 
neither researchers nor policy makers have the practical or theo- 
retical expertise to design assessment mechanisms without consid- 
erable input from other disciplines and sectors that have more reg- 
ularly incorporated evaluation procedures into their operation. In 
particular, the Committee could identify and seek advice from cor- 
porations that have included performance assessment as a part of 
their research programs.*° 

As an ancillary part of the performance assessment process, the 
Committee could establish programs to test and compare alterna- 
tive methods of research administration and performance, such as: 

¢ Peer review of grant proposals vs. alternatives such as block 
grants, grants based on past performance, start-up grants for 
young researchers, funding decisions by “smart managers,” 
etc.; 


°® Ronald D. Brunner, “Performance as promised: Restructuring the civil space program,” 
Space Policy 8, May, 1992, pp. 116-136; Rubin et al., “Keeping Climate Research Relevant,” op. 
cit., footnote 2. 

5° There are some initial efforts underway; see, for example, Ronald R. Kostoff (Director, 
Technical Assessment, Office of Naval Research), “Research Impact Assessment,” presented at 
ae ahh International Conference on Management of Technology, February 17-21, 1992, 

iami, FL. 

*° Performance assessment must integrate all criteria which bear on the ability to achieve 
goals; failure to achieve goals can be the fault of unrealistic goals, unfocused research, and/or 
failure to adequately link the research process to the implementation process. See Eleanor Che- 
limsky, “Expanding GAO’s Capabilities in Program Evaluation,” The GAO Journal, Winter/ 
Spring 1990, pp. 43-52; Thomas Cook and William Shadish, Jr., “Program Evaluation; The 
Worldly Science,” Annual Reviews of Psychology, vol. 37, 1986, pp. 193-232; and Carol H. Weiss 
(ed.), Organizations for Policy Analysis (Newbury Park, CA; Sage Publications, 1992). For brief 
discussions of corporate approaches to performance assessment in research, see Michael 
Schrage, “When Innovation Leads Corporations Away From Their Strategic Interests,” Wash- 
ington Post, May 31, 1991; and Testimony by C. Kumar N. Patel before the Subcommittee on 
Science, Committee on Science, Space, and Technology, Hearing on “Setting Priorities in Sci- 
ence,” April 7, 1992. 
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e Individual-investigator and small-team research vs. research 
aggregated and administered through centers of excellence or 
other larger units; 

e Disciplinary vs. interdisciplinary approaches to research 
problems; 

e National vs. international research teams; 

e Mission-oriented research conducted at intramural, vs. extra- 
mural, laboratories; 

e Basic research conducted at universities, national laborato- 
ries, and corporate laboratories; 

e “Scientific excellence” vs. more restrictive funding criteria 
(i.e., excellence plus additional criteria such as innovativeness, 
educational potential, policy-relevance); 

e Other more creative approaches. 

Some of these comparisons could be conducted by analysis of ex- 
isting data, and performed by organizations such as the Office of 
Technology Assessment. Others may require that the Committee 
authorize research agencies to set aside certain portions of their 
budgets to support alternative research approaches and assess rela- 
tive effectiveness. Overall, the Committee should begin to test the 
science policy assumptions that have guided its decision making, 
and develop new models governing science policy that are respon- 
sive to the needs of the nation as we move toward the twenty-first 
century. 


CONCLUSIONS AND PROSPECTS 


This document provides a first rationale for rethinking federal 
research policy. As part of this process, policy makers must look 
beyond the traditional categories that dominate research policy 
dialogue: basic versus applied; research versus development; sci- 
ence versus technology; knowledge creation versus knowledge diffu- 
sion; big science versus little science. The Committee should strive 
to develop a ‘systemic approach to research policy which is not re- 
stricted by artificial tradeoffs and vested interests. 

Recommended policy action focuses on creating better linkages 
between research performance and national goals. Federal policy is 
one (but not the only) avenue for building and strengthening such 
linkages. Two nested areas of. systemic policy action are recom- 
mended: developing better mechanisms for government-wide sci- 
ence-policy making; and establishing a system of performance as- 
sessment that is integrated within the research system. These rec- 
ommendations are strategic in nature. They are intended to pro- 
vide a long-term context for Committee action leading to improved 
integration of the research and policy processes, and more effective 
application of research to the problems of society. 
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